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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36 (a). In no event, however, may a reply be timely filed after SIX (6) MONTHS from the 
mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)D Responsive to communication(s) filed on 

2a) □ This action is FINAL. 2b) 53 This action is non-final. 

3) □ Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1; 453 O.G. 213. 
Disposition of Claims 

4) B Claim(s) 1-42 



4a) Of the above, claim(s) 
5)D Claim(s) 



6) 53 Claim(s) 7-42 

7) D Claim(s) 

8) D Claims 



is/are pending in the application, 
is/are withdrawn from consideration. 

is/are allowed. 

is/are rejected. 

is/are objected to. 



are subject to restriction and/or election requirement. 



Application Papers 

9)D The specification is objected to by the Examiner. 

10)§3 The drawing(s) filed on Nov 24, 2000 is/are a) jx) accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
1 1 )□ The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner 

If approved, corrected drawings are required in reply to this Office action. 

1 2) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgement is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some* c)D None of: 

1. □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 



3. □ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
*See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. § 1 19(e). 
a)D The translation of the foreign language provisional application has been received. 

15) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. §§120 and/or 121. 
Attachment(s) 

1 1 j)3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application {PTO-1 52) 

3) 53 Information Disclosure Statement(s) (PTO-1 449) Paper No(s). 4,5,6 3) Q other: 
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DETAILED ACTION 



Claim Rejections - 35 U.S.C. § 102 



1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless -- 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371© of this title before the invention 
thereof by the applicant for patent. 

2. Claims 1-5, 10-24, 26-29 and 31-33, are rejected under 35 U.S.C. 102(e) as being 
anticipated by Maki et al (U.S.6,072,903). 

Regarding claim 1, Maki et al disclose a system, comprising a computer configured to 
determine a position and shape of an object of interest from video images and characterize 
activity of said object of interest based on analysis of changes in said position and said shape 
over time, (see column 15, 49-57, line refer to detecting information about position, shape and 
posture of object). 

Regarding claim 2, Maki et al disclose the system of claim 1, further comprising: a video 
camera coupled to said computer for providing said video images, (see column 23, line 64 throw 
column 24, line 7, tracking of motion picture; by use of television camera). 

Regarding claim 3, Maki et al disclose the system of claim 2, further comprising: a video 
digitization unit couple to said video camera and said computer for converting said video images 
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provided by said video camera from analog to digital format, (see above claims and column 8, 
line 46-51, refer to using a digital filter). 

Regarding claim 4, Maki et al disclose the system of claim 3, further comprising: a 
storage/retrieval unit coupled to said video digitization unit, said video camera, and said 
computer, for storing said video images and standard object video images, (see column 25, line 
32-37, refer to storage section). 

Regarding claim 5, Maki et al disclose the system of claim 1, wherein said computer 
includes an object identification and segregation module receiving said video images, (see 
column 29, line 23-28, refer to receiving three-dimensional position information). 

Regarding claim 13, Maki et al disclose the system of claim 1, wherein said object is a mouse, 
(see column 2, line 20-26 refer to three dimensional mouse). 

Regarding claim 19, Maki et al disclose the method of claim 18, wherein said step of 
generating a background image includes the step of determining variation in intensity of pixels 
within said individual frames to identify a region where said foreground object is located, (see 
column 19, line 44-52, corresponding to pixel and intensity of the light). 

Regarding claim 26, Maki et al disclose the method of claim 24, wherein said step of 
identifying and classifying changes to said foreground object uses contour-based shape 
information selected from the group consisting of b-spline representation, convex hull 
representation, and corner point, (see column 9, line 14-23, refer to connected feature points, 
such as outer corner of the eye ). 
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Regarding claim 27, Maki et al disclose the method of claim 24, wherein said step of 
identifying and classifying changes to said foreground object includes identifying a set of model 
postures and their description information, said set of model postures including horizontal 
posture, vertical posture, eating posture, or sleeping posture, (see column 25, line 6-11, refer to 
posture information). 

With regard to claims 10-18, 20-24, 28-29 and 31-33, the arguments analogous to those 
presented for claims above are applicable. 



3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 6-9, 25 and 30, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Maki et al (U.S.6,072,903) in view of Crabtree et al (U.S.6,263,088). 

Regarding claim 6, Maki et al disclose the system of claim 5, wherein said object 
identification and segregation module operates using a background subtraction algorithm in 
which a plurality of said video images are grouped into a set, a standard deviation map of the set 
of video images is created, a bounding box where a variation is greater than a predetermined 
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threshold is remove from said set of video images, and the set of images less said bounding 
boxes is averaged to produce a background image, (see column 25, line 45-52, refer to specific 
threshold value and extracting of connected feature point). 

However Maki et al does not explicitly state " standard deviation " . On the other hand 
Crabtree et al teaches (see column 17, line 61-64, the standard deviation value for the region 
cluster density, "the number of pixels in a region cluster area in the oriented bounding box" are 
provided). 

Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Maki et al invention according to the teachings of Crabtree et al 
because it is method to measure dispersion of a group of measurements relative to the 
mean(average) of that group or defined as the square root of the expected value of the square of 
the difference between a random variable. 

Regarding claim 8, Maki et al disclose the system of claim 7, wherein said-computer 
further includes an object tracking module for tracking said object from one frame of said video 
images to another frame, and an object shape and location change classifier for classifying the 
activity of said object, coupled to each other, said object identification and segregation module, 
and said behavior identification module, (see column 25, line 1-1 1, refer to tracking the motion 
of object, according to the position of feature points extracted at each frame). 

Regarding claim 9, Maki et al disclose the system of claim 8, wherein said computer 
further includes a standard object behavior storage module that stores information about known 
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behavior of a predetermined standard object for comparing the activity of said object, said 
standard object behavior storage module being coupled to said behavior identification module, 
and a standard object classifier module coupled to said standard object behavior module, (see 
column 27, line 20-3 1, refer to images comparison). 

With regard to claims 7, 25 and 30, the arguments analogous to those presented for 
claims above are applicable. 

5. Claims 34-42, are rejected under 35 U.S.C. 103(a) as being unpatentable over Maki et al 
(U.S.6,072,903) in view of Crabtree et al (U.S.6,263,088) and Smith et al (U.S.5,870,138). 

Regarding claim 34, Maki et al disclose the method of claim 33, wherein said temporal 
analysis is a time-series analysis such as Hidden Markov Model (HMMs), (see above claims). 

However Maki et al and Crabtree et al does not explicitly state " Hidden Markov Model " 
. On the other hand Smith et al teaches (column 17, line 41-43, the device can be implemented 
as a Hidden Markov Model and has two phases of operation, namely training and recall). 

Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Maki et al and Crabtree et al invention according to the teachings 
of Smith et al because it is device to map the input data (images) from the camera to outputs 
which represents the probability of the input images belonging to a specified set of expressions. 



With regard to claims 35-42, the arguments analogous to those presented for claims 
above are applicable. 
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Other prior art cited 

6. U.S. patent 5,596,994 to Bro is cited for automated and interactive behavioral and 
medical guidance. 

Contact Information 

7. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Seyed Azarian whose telephone number is (703) 306-5907. 

The examiner can normally be reached on Monday through Thursday 6:00 a.m. to 6:30 p.m. If 

attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor,Leo 

Boudreau, can be reached at (703)305-4706. 

Any response to this action should be mailed to: 

Assistant Commissioner for Patents 
Washington, D.C. 20231 
or faxed to: 

(703) 306-0377, ("draft" or "informar'communications should be clearly labeled 
to expedite delivery to examiner) 

Hand delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA., Sixth Floor (Receptionist). 



Any inquiry of a general nature or relating to the status of this application should be directed to 
the T.C. customer service office whose telephone number is (703) 306-0377. 




Group Art Unit 2621 
May. 13,2002 



